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(57) Abstract: A method and arrangement are disclosed 
for distributing multimedia content such that that the ac- 
tual distribution of said content can be monitored in an ef- 
ficient and reliable manner. The invention combines the 
extensiveness of feature extraction and the robustness of 
watermarking. Characteristic features of the content, e.g. 
luminance distribution, are extracted (11) to constitute a 
signature (SIG) of the content. In addition, a watermark 
is embedded (12) having a payload representing an index 
(ID) in a database (10) in which the content to be moni- 
tored is stored. The watermark serves as an index for lim- 
iting the database search needed for monitoring the sig- 
natures. 
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MULTIMEDIA MONITORING BY COMBINING WATERMARKING AND CHARACTERISTIC 

SIGNATURE OF SIGNAL 



FIELD OF THE INVENTION 

The invention relates to a method and a system for distributing media content 
such as audio and/or video programs. The invention also relates to a method and a system for 
monitoring the distribution of such media content. 

5 

BACKGROUND OF THE INVENTION 

Producers of audiovisual content, such as television broadcasters or 
advertisers, are often interested in having knowledge as to whether, where and when their 
work is distributed. To this end, "broadcast monitoring" systems have been developed 
10 recently. 

In one known broadcast monitoring method, sometimes referred to as active 
monitoring, a watermark is embedded in the video content The payload in the watermark is a 
pointer to a database entry corresponding to the video sequence (e.g. a clip, a movie scene, a 
commercial). The watermark is retrieved and used to identify the content. A problem of this 

15 method is that a large payload is necessary for exploiting a large database. Such a large 
payload is difficult to embed in an imperceptible and unobtrusive manner. 

In another known broadcast monitoring method, referred to as passive 
monitoring, so-called robust features are extracted from the content. Robust features are 
variables computed from the content, which remain more or less xmchanged as long as the 

20 scene does not change too much. For every scene there is a unique set of robust features. For 
example, video pictures are divided into blocks. Each block is represented by a bit indicating 
whether the Ixmainance of said block is higher or lower than a given reference value (e.g. the 
luminance of the neighboring block). The bit string corresponding to the robust features is 
then used as a signature for the particular scene in the database. A problem of this method is 

25 that the bit string extracted at the monitoring end may be slightly different from the signature 
in the database. This makes it difficult to search the signature in the database which most 
closely resembles the extracted bit string. Moreover, the signature is not necessarily unique. 
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OBJECT AND SUMMARY OF THE INVENTION 

It is an object of the invention to improve the robustness and efficiency of 
content monitoring. 

To this end, the invention provides a method and a system for distributing 
5 media content and for monitoring said media content as defined in the appended claims. 

The current invention is an inventive combination of the prior-art schemes. 
The media content is watermarked with a relatively small payload giving information on that 
section of the database in which the content is to be searched. The robust signature is then 
used for a search within said section. The embedded watermark thus serves as an index for 
1 0 restricting the database search. It is achieved with the invention that the embedded watermark 
can be relatively small and thus robust. At the same time, the database search problem is 
reduced in complexity. 

For example, the database may contain scenes of a large number of movies. 
The watermark indicates which movie a given scene belongs to. The extracted signature is 
1 5 then used to find the particular scene from that movie. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 shows a schematic diagram of a system for distributing media content in 
accordance with the invention. 
20 Fig. 2 shows a schematic diagram of a system for monitoring the distribution 

of said media content in accordance with the invention. 



DESCRIPTION OF EMBODIMENTS 

In Fig. 1, a signal X represents media content, for example, a video scene. The 

25 video scene may be a particular news item produced by the BBC, or a commercial of 
company A. The BBC wants to trace broadcasts of its news item by other broadcasters. 
Company A wants to check whether its expensive commercial has indeed been broadcast at 
prime time through the TV stations of network B. Similar considerations apply to other 
distribution networks such as the Internet. 

30 At the distribution end the system comprises a database 10, a feature extractor 

11, and a watermark embedder 12. The feature extractor 11 extracts a signature SIG from the 
actual video content. For example, the video pictures are divided into blocks. Each block is 
represented by a bit indicating whether the luminance of said block is higher or lower than 
the luminance of the neighboring block. However, it will be appreciated that an infinite 
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nximber of alternative embodiments of feature extractor 11 can be designed by a person 
skilled in the art. Similar features can be extracted from an audio signal, for example, a 
digitized version of the actual frequency spectrum. 

The bit string thus generated by the feature extractor 11 constitutes a signature 
5 SIG. The signature SIG is applied to the database 10 and stored in a field of a record 

corresponding to the news item or commercial. The relevant record in the database is given a 
unique ID. Said ID is applied to the watermark embedder 12 and encoded as payload of a 
watermark. An example of such a watermark embedder having the capabilit}'^ to convey a 
multi-bit payload is disclosed in Intemational Patent Application WO 99/45705. The 

10 watermarked content X' is then broadcast or otherwise distributed. 

The arrangement shown in Fig. 2 monitors the relevant distribution channel. It 
receives the content X* and comprises a watermark detector 21 and a feature extractor 22. The 
arrangement is coupled to the database 10. Note that this is not necessarily a real-time 
connection. The watermark detector 21 detects the embedded watermark and, if the 

15 watermark is found, decodes its payload ID. An example is disclosed in Intemational Patent 
Application WO 99/45705. The feature extractor 22 performs the same operation as feature 
extractor 11 in Fig. 1 . It will be appreciated that the extractor is designed in such a way that 
the robust features of the content are not affected by the embedded watermark. 

The decoded watermark payload ID and the signature SIG are applied to the 

20 database. Because the watermark provides a reliable data channel, the decoded payload may 
be assumed to be identical to the ID embedded by the arrangement which is shown in Fig. 1 . 
The search for the signature SIG in the database (or the search for a closely resembling 
signature in view of possible changes caused by processing and transmission) can be limited 
to those records having the relevant ID. In database terminology: the ED serves as an index 

25 for limiting the database search. 

A metihiod and arrangement are disclosed for distributing multimedia content 
such that the actual distribution of said content can be monitored in an efficient and reliable 
manner. The invention combines the extensiveness of feature extraction and the robustness of 
watermarking. Characteristic features of the content, e.g. luminance distribution, are 

30 extracted (1 1) to constitute a signature (SIG) of the content. In addition, a watermark is 

embedded (12) having a payload representing an index (ID) in a database (1 0) in which the 
content to be monitored is stored. The watermark serves as an index for limiting the database 
search needed for monitoring the signatures. 
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1 . A method of distributing media content, comprising the steps of extracting 
features of said media content, storing a signature representing said features in a section of a 
database, embedding a watermark in said media content indicative of said section of the 
database, and distributing the watermarked media content. 

2. A method of monitoring distribution of media content, comprising the steps of 
receiving said media content, detecting a watermark in said media content indicative of a 
section of a database, extracting features of said media content, and searching a signature 
representing said features in said section of the database. 

3. A system for distributing media content, comprising means for extracting 
features of said media content, means for storing a signature representing said features in a 
section of a database, means for embedding a watermark in said media content indicative of 
said section of the database, and means for distributing the watermarked media content. 

4. A system for monitoring distribution of media content, comprising means for 
receiving said media content, means for detecting a watermark in said media content 

indicative of a section of a database, means for extracting featmes of said media content, and 
means for searching a signature representing said features in said section of the database. 
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